Tomáš Pospíšek's Notizblock

Exporting data from a Lumia smartphone

Once again Microsoft have positively surprised me. I feel that company is changing for the better. Sure the Lumia 535 is crap:

it reboots at least once a week. It did that once at the first ring when someone was calling.
I can barely manage to write an SMS with its on screen keyboard. Emails are completely out of question.
selecting/copy/pasting is an exercise in self frustration
OneNote one the Smarphone is very painful to use. Why are list entries wider than the screen?

But! Microsoft has allowed me to easily export all the data I wanted from the phone.

That includes all SMS'es and all MMS'es.

Microsoft lets you do that with their contacts+message backup app. OK, they'll hide it under Settings->Extras->contacts+message backup. And yes it'll crash when you try to export contacts, SMS and MMS'es all at once.

But once that's circumnavigated, it will let you select the place to backup to and will export the data in a nice plaintext XML file.

Here's how I transform the SMS export into something human readable:

#!/bin/bash
#
# Input File as STDIN

xsltproc ArrayOfMessages_to_txt.xsl -     | perl -n -e '
use POSIX qw/strftime/;

s/\+41791234567/Eddy/;
s/\+41797654321/Grant/;

if(    /^From: *$/) {}
elsif( /^To: *$/) {}
else {

  s/^From: /<- /;
  s/^To: /-> /;
  s/^Text: /   /;

  if( /^LocalTimestamp: (.*)/ ) {
    print( "   (".(strftime( "%y-%m-%d %H:%M", localtime( $1 / 10000000 - 11644473600 ) ) ).")\n");
  }
  else {
    print $_;
  }
}

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes"/>
    <xsl:output method="text"/>
    <xsl:template match="ArrayOfMessage">
        <xsl:for-each select="Message" xml:space="preserve">
From:<xsl:value-of select="Sender"/>
To:<xsl:value-of  select="Recepients/string"/>
LocalTimestamp:<xsl:value-of  select="LocalTimestamp"/>
Text:<xsl:value-of  select="Body"/>
<xsl:text>&#xa;</xsl:text>
        </xsl:for-each> 
    </xsl:template>
</xsl:stylesheet>

That will give you something like:

<- Eddy
   (2016-11-05 18:20)
   Hello how do you like my old songs?

-> Eddy
   (2016-11-05 18:21)
   Oh, very pleased to meet you. I like them a lot.
   But haven't you been dead for a long time? :-(

Extracting MMS pictures:

#!/bin/bash
#
# Input File as STDIN

xsltproc Extract_pics_from_mms.xsl - | csplit -f base64 - /^$/ {*}
for i in base????; do
  cat "$i" | uudecode > "$i.decoded"
done

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes"/>
    <xsl:output method="text"/>
    <xsl:template match="ArrayOfMessage">
        <xsl:for-each select="Message/Attachments/MessageAttachment/AttachmentDataBase64String">
            <xsl:value-of select="."/>
<xsl:text>&#xa;</xsl:text>
<xsl:text>&#xa;</xsl:text>
        </xsl:for-each> 
    </xsl:template>
</xsl:stylesheet>

(Yes, that code has only hack quality. Inspect it before using it!)

So wow Microsoft, thanks a lot, you made my day!

However while hacking on that XSLT transormation it occured to me that:

XML is a useful data transfer convention
XML is still eye cancer inducing
while powerful, XSLT is a waste of time and a time sink

I would like to reflect the last of those assertions. I am only a lowly Unix and network sysadmin and so I only seldom do data transformations. So what I have to say about XSLT might not apply to someone who does data transformation for a living.

For an irregular XSLT user, that tool seems to be way, WAY too archaic. I mean - to insert a newline you have to write:

<xsl:text>&#xa;</xsl:text>

WHAT

THE

FUCK

More or less trivial things, like date and time formatting lead to hours of googling and trying before finding out that it only works as presented in various Stackoverflow discussions with Java XSLT processors...

So I think implementing the transformation via XSLT was a mistake. Using XPath grammar to select elements makes a bit sense, because that knowledge can be reused with browser and DOM technology. But for all transformations of the values inside that XML text it would be much better to find tooling provided by a friendly programming language, like Ruby, Python, etc.

There one could use XPath for node selection, maybe some specialized facility for iteration and node cretion, but certainly the usual nice progamming facilities for value transformation.

Tomáš Pospíšek, 2016-11-04